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Abstract 



o 

>--^ ' We propose a novel approach to modeling advertising dynamics for a firm operating over 

distributed market domain based on controlled partial differential equations of diffusion type. 



^ ' Using our model, we consider a general type of finite-horizon profit maximization problem in 

Qa monopoly setting. By reformulating this profit maximization problem as an optimal control 
problem in infinite dimensions, we derive sufficient conditions for the existence of its optimal 
^O . solutions under general profit functions, as well as state and control constraints, and provide 

general characterization of the optimal solutions. Sharper, feedback-form, characterizations 
r ) • of the optimal solutions are obtained for two variants of the general problem. 
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In the last two decades, dynamic optimal control models have become a mainstay in the market- 



1 Introduction 
(N 
> 

^r^ , ing literature focused on advertising. While the empirical marketing research pays increasingly 

'nI" I growing attention to the multiple-market aspects of the advertising strategy ([7], [10]), the mod- 

eling support for the studies of advertising policies distributed over multiple geographic regions 
T^lj- , has been lacking (see [TT] for the most comprehensive review of the related literature) . In the 

^P I present paper, we propose a simple model which deals with both the dynamic as well as the 

^ ■ spatial effects of advertising. 

^^ I 

C^ . The classical paper by Nerlove and Arrow [16j introduced the following model for the dynam- 

ics of goodwill stock under the influence of advertising for a monopolist in a single market-single 
product environment: 

X' ^ = u{t)-px{tl (1) 

Cd ' where x{t) is the goodwill level at time t > 0, u is the rate of advertising (in monetary terms), 

and /? > is a constant factor describing the deterioration of the goodwill in the absence 
of advertising. Our analysis extends the Nerlove-Arrow (NA hereafter) model by considering 
spatially distributed advertising and by modeling geographic fluctuations in the goodwill stock. 
The treatment of problems of optimal advertising in the space-time setting is not new: to the 
best of our knowledge it was considered for the first and only time in [18] as a special case of 
a very general class of distributed-parameter optimal advertising models, where the parameter 
space is modeled as subset of a measurable space. The aim of |I8J was to establish a general, 
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abstract framework and essentially proved the well-posedness of the problem and the existence 
of optimal strategies under suitable assumptions. Our purpose here is different, as we focus 
instead on a simple NA model with a diffusive component, for which optimal strategies can be 
better characterized, sometimes even explicitly. 

In the first part of this work we address a modeling question focused on describing the 
dynamics of goodwill which depends on both space and time coordinates. In particular, we 
propose to model the dynamics of goodwill through the following controlled partial differential 
equation (PDE) 

— (t,0 = -pxit,^) + A^xit,0+b{Ouit,0 (2) 

(with appropriate initial and boundary conditions - see ([!]) below), where x : [0,T] x H ^ M, 
H C M^ is the goodwill "density", u(-,-) is the rate of advertising effort (expressed in Gross 
Rating Points, or GRPs), and b{-) is the coefficient of advertising effectiveness. The second 
term on the right-hand side of ^ is introduced to capture the effect of the spatial diffusion of 
goodwill (more detailed rationale for the use of this term is described in the next section) and 
reflects the main modeling difference between our analysis and that of Nerlove and Arrow [16] . 
The last term on the right-hand side of ([2]) represents another new feature of our model: we 
describe advertising effort using GRP parameter, which recent trend in advertising literature 
([IS], [ID]) defines as more appropriate than the traditional expenditure rate. Note that ^ 
reduces to the NA model if x does not depend on the spatial coordinate ^ and if 6(^) = 1. 
Moreover, we shall see that the NA model is also recovered by averaging in space our model. 

In the second part, we formulate and solve the space-time analogs of some of the optimal 
control problems studied earlier for the cases where x depends only on time. Most of the results 
will follow using the tools of optimal control in infinite dimensional spaces. We should clearly 
state at this point that the emphasis in the paper is not on deriving new results in infinite 
dimensional optimal control, but rather on showing that models and techniques of analysis and 
control in Hilbert spaces could prove useful to extend existing advertising models to account for 
geographic fluctuations. Moreover, we devote some effort to show that in some simple, but still 
interesting situations, optimal policies can be obtained in closed form, and are hence amenable 
to practical implementation. Finally, we discuss how the diffusive term influences the optimal 
strategy by a direct comparison with the standard NA model. 

In particular, we consider the following types of problems: in the most general case our aim is 
to maximize a monopolistic firm's finite horizon profit expressed as a combination of a functional 
of the terminal goodwill stock and the cumulative cost of advertising. We approach this problem 
in the abstract setting of control of infinite-dimensional systems via Pontryagin's maximum 
principle: under mild assumptions on the structure of the problem, one can obtain quite general, 
although abstract, results on the existence and characterization of optimal policies. For some 
specific choices of cost and reward functions, optimal policies can be expressed in closed form. 
A dynamic programming approach allows one to characterize the value function as solution of 
an Hamilton-Jacobi equation, and to express optimal strategies in feedback form. In the special 
case of quadratic cost and reward functions, a linear-quadratic (LQ) regulator approach yields 
explicit expressions for optimal strategies in feedback form. The LQ approach is also used to 
solve a related problem, i.e. the minimization of a weighted sum of the distance of x{T, •) from 
a target goodwill level and the total cost of reaching the target. 

Our analysis follows a classical scheme: first we establish that the controlled PDE ^ can 
be written as an abstract linear control system of the type 

^ = Ax{t) + Bu{t) (3) 



in a suitable Hilbert space of functions X. Then we show that the control problem for the 
PDE ([2]) is equivalent to a control problem on X for the abstract differential equation ([3]). In 
the general case, the optimal advertising problem is solved (although only in an abstract way) 
using the weak maximum principle in infinite dimensions, the theory of which can be found, 
e.g., in [2]. The simpler case of quadratic utility and cost functions is reduced to the study of 
a Riccati equation by an application of the dynamic programming principle. The solution of 
such an equation yields, in particular, a feedback-type optimal policy for the original problem. 
For the infinite-dimensional LQ problem we refer to ^, ^, ^ for the standard case of positive 
definite costs (see also [l3]), and to [H] for the general case with indefinite costs. 

The rest of the paper is organized as follows: section [2] formally defines and motivates the 
model for the dynamics of goodwill as a function of space and time, as well as the associated 
optimal advertising problems. The model as well as the optimization problems are recast in 
the framework of control systems in Hilbert spaces in section [3l Sections 4 to 7 deal with the 
optimal control problems mentioned above, and section 8 concludes. 

2 Model for spatial advertising dynamics 

Let us first introduce some notation used throughout the paper. Let X be a real separable 
Hilbert space with norm | • | and inner product (•,•). We denote by L2([0, T],X) the Hilbert 
space of functions / : [0,T] ^ X with finite norm |[/||, 



[ \f{t)\'dt <oc. 
Jo 



When there will be no possibility of confusion, we shall still denote by (•, •) the inner product 
of spaces like L2{[0,T],X). The space of linear bounded operators between the Hilbert spaces 
X and Y will be denoted as C{X, Y), or simply as C{X) when X = Y. 

Given an operator A : D{A) C X — > X, we shall say that A is uniformly positive definite 
(denoted A :> 0) if there exists e > such that A — eI>0, i.e. {Ax,x) > e|xp for all x G D{A). 
The adjoint of an operator A is denoted by A* . 

We shall need some standard functional spaces: for a bounded open set Q, C M'^, H'^{Q,) is 
the Sobolev space of functions with (generalized) derivatives up of order k in L2(^), and Hq{Q) 
is the closure of Cq°(0) in the topology of H^{il,). Here Cq^{U) stands for the space of infinite 
differentiable functions with compact support on fi. 

The following elementary notions from convex analysis will be used: given a Hilbert space 
X and convex lower semicontinuous function f : X ^ — oo,-|-oo], the mapping df : X ^> X 
defined by 

df{x) = {wGX:{x-y,w)> f{x) - f{y)yy E X} 

is called the subdifferential of /. Moreover, the conjugate (or Fenchel-Legendre transform) of / 
is the function /* : X — >] — oo, +cx3] defined by 

f*{p) = sup((p,x) - f{x)). 

xex 

Let S be a bounded open set of M^ with regular boundary dE. This set will be the model 
for the geographic region of interest for the advertising campaign. Let x : [0, T] x H — > M be 
the goodwill "density" of a given product, specified as a function of time t and location .^ E H. 



The model for the controlled dynamics of x will be given by the following equation: 

(JX 

|^(t,0=o, it,Oe]0,T]xdE, (^^ 

where T > is a fixed time (which could be thought, for instance, as the time of market 
introduction for the new product), p > is a natural deterioration factor of the product image 
in absence of advertising, A^ = -^ + -^ is the Laplacian with respect to the spatial variable 
S,, u : [0,T] X H ^ M+ is the advertising effort, which also depends on time and location, and 
6 : H ^ R-i_ is a (bounded) factor of advertising effectiveness, xq : H — > M represent the initial 
level of goodwill as a function of location in the region of interest (for example, one may assume 
xo = if at time t = the advertised product is unknown). 

The introduction of the diffusion term A^x{t, ^) is motivated by an analogy with the classical 
diffusion equation, used to model the evolution in time of the concentration of a substance in 
an homogeneous medium. The idea is to regard a consumer population in a bounded region H 
as the medium in which the goodwill is spreading. The Neumann boundary condition in ([3j) 
is equivalent to assuming that there is no exchange of goodwill through the boundary of the 
region S, i.e. that the consumer population inside H does not communicate with the outside 
world. This assumption is natural (at least when considering large enough populations, such 
as a nation) and allow us to recover the classical NA by averaging our model in space. Indeed, 
integrating ^ we get 

The total goodwill in the region H is given by x{t) := f^ x{t, S,) dS,- Set u{t) = J„ b{S,)u{t, ^) d^ (in 
the special case 6 = 1 this would be the total advertising effort). Note that, by the Gauss-Stokes 
formula and the Neumann boundary conditions in dH), one has 

^ Ax{t, Od^ = 1^ Ax(t, om d^ = j ^ vx(t, e)vi(e) d^ = o. 

Hence, by changing the order of differentiation and integration on the left hand side, we obtain 

dx 

which represents the classical NA dynamics for the total goodwill x(-). 

Remark 1. (i) Equations like (JH) have also been successfully used to model the dynamics of 
populations, as well as the dynamics of infective diseases, in bounded regions in M? or R^. Let 
us also recall that simple epidemics models (without spatial dimension) were the starting point 
for Bass' model of the diffusion of new goods, which still is a reference model in the marketing 
and management science literature. These considerations, together with the above ones, would 
hopefully give a plausible foundation to our model. 

(ii) The concept of "goodwill density" is not at all difficult to interpret nor too abstract. 
In fact, it can be (roughly) interpreted as the ratio of total goodwill over a region and the 
(2-dimensional Lebesgue) measure of the region, provided the region is small enough. 

(ii) In the case of a non- homogeneous consumer population the term Ax(t, ^) modeling 
goodwill diffusion would be replaced by a more general second-order elliptical operator, adding 
only some technical complications. 



Below we will use the controlled dynamic model ^ to identify the optimal advertising 
strategy for a monopolistic firm. Traditional advertising literature provides several choices in 
selecting the objective to be maximized in such setting. In the present work we adopt the 
perspective of a firm which is prepared to launch a new product or service at a pre-determined 
time T in the future and which aims at maximize its total pre-launch profits. Therefore, a rather 
general distributed advertising problem one would like to solve can be described as follows: 

(P) maximize the functional 

Mu) = ljo(.x{T,0)d(- I l^ho{u{t,0)d^dt (5) 

over all controls u(t,^) € [0,R] (t, ^)-a.e., subject to the dynamics ^ and the additional 
constraint x(t,^) > (t,^)-a.e. 

Here 0o, ho : M ^ M are such that the integrals in ([5]) are finite, and can be thought as utility 
(profit projection) of final goodwill and unit cost of pre-launch advertising, respectively. 

We shall see that if the cost function /iq is quadratic, an alternative approach based on 
Hamilton-Jacobi equations allows one to obtain a feedback characterization of the optimal 
strategy. Quadratic functions provide the simplest (strictly) convex cost functions and are 
commonly used (see e.g. [IH]). 

We shall also consider two simpler variants of the general advertising problem for which 
sharper characterizations of the optimal advertising policy can be obtained. The first one, 
which we denote as (PI), is formulated as follows: 

(PI) maximize the functional 

Ji{u)=^ jjx{T,i)\^di- j jju{t,^)\^didt (6) 

over all controls u S L2([0, T] x H,M), subject to the dynamics (jl]). Here the weight 
coefficient 7 > represents the relative impact of the profit contribution of the final 
goodwill level vs. that of the advertising effort. 

While a quadratic cost of advertising effort is a common and plausible assumption, the choice 
of this specific form for the profit contribution of the final pre-launch goodwill level in ([6]) is 
not an obvious one. In particular, it is not concave, and as such it can be interpreted as the 
utility function of a risk-loving firm. However, in some situations this assumption is reasonable 
(see e.g. [12] for a related discussion) and can also be justified by considerations of (local) 
second-order approximations and of analytical tractability. 

In addition to (PI) we also study a "targeting" problem denoted as (P2): 

(P2) minimize the functional 

Jk{u) = l jjx{T,0-KO?di + j lju{t,0\^d^dt (7) 

over all controls u E L2([0, T] x S, M), subject to the dynamics (H]), where 7 > is a weight 
coefficient as above, and A; : H — > M is the target distribution of goodwill at time T. 

The targeting problem may arise in settings where a firm would like to evaluate the resources 
required for establishing a particular goodwill profile k{S,) for the product to be launched. In 
particular, if a firm is interested in enforcing a uniform good will profile A:(^) = /cq, setting 7 
appropriately high and solving the targeting problem repeatedly for several values of /cq will 
provide an assessment of necessary advertising spending as a function of /cq- Thus, in essence, 
the targeting problem serves as a surrogate penalty-function formulation of the problem in 
which the final goodwill profile is set to equal k{^). 



3 Reformulating the spatial advertising problem as a control 
problem in infinite dimensions 

Let X be the Hilbert space of square integrable functions defined on the domain H, i.e. X = 
L2(H), equipped with the natural inner product 

and norm 

l/h=( / fiOdi 



Denote by A the following linear operator in X: 
{ Ay = {A- p)y, 

D{A) = (ve H^{E) : Av G L2(H), / Av(f)dx = - f {Vv,V^) dxV 

Note that, in view of Green's formula, the second condition defining the domain of ^ in ([8|) can 
be seen as a weak formulation of the Neumann boundary condition dv/dn = on dr.. 

Setting (with a slight abuse of notation) x{t) = x{t, •) and u{t) = u(t, •), we can equivalently 
write dU as an abstract linear system on the Hilbert space X: 

-^ = Ax(t) + Bu(t), 

dt "■ ' ^ ^' (9) 

2;(0) = xq^ X 

for t € [0, T], with A : D{A) C X ^ X as in 1^, u £ L2([0, T],X), and B : X ^ X is the linear 
bounded operator defined by 

B : y{0 ^ b{^)y{0. (10) 

We shall need the following important features of the Neumann Laplacian, which we collect in 
the form of a lemma. The proof can be found e.g. in [17] and |j20j. 

Lemma 2. The linear operator A defined in ^ is self-adjoint, negative, and generates a 
strongly continuous positivity-preserving semigroup of contractions. Moreover, the Cauchy prob- 
lem (^ admits a unique mild solution x given by the variation of constants formula: 



x{t) = e'^xo + / e(*-^)^Bu(s) ds, (11) 

Jo 



'0 

where e denotes the strongly continuous semigroup generated by A 
Problem (P) can now be written as 

rT 



inf L(x(r))+ / hi{u{t))dt] (12) 

«e" \ Jo J 

subject to the dynamics ([9]), where 0, /ii : X — > M are defined as 

0(x) = - jj^{x{i))di, (13) 

hi{u) = f ho{u{0)d^, 



and 

il = {n : [0,r] ^ X\ u{-m G [0,i?], x(-)(e) > ^-a-e.}. 

Similarly, the objective functionals © and ([7]) can be respectively written as 

J.(n) = -7|x(r)|2+ / |n(t)|2(ii 

JO 

and 

J;,(u) = 7!x(r) - A:|2 + f \u{t)\^dt. 

Jo 

The aim is to find an optimal control, i.e. a function n* E il^rf such that 

J(n^,) < J{u) Vn e ilarf. 

Here J is either Jc, Jj, or J/j, and Had is the class of admissible controls: Had = ^ for problem 
(P), and Had = L2{[0,T],X) for problems (PI) and (P2). The pair (x*,ii*), where x* is the 
solution of Q with u = n^,, is often called the optimal pair for the corresponding optimal control 
problem. 

Remark 3. Problem (PI) is a linear-quadratic (LQ) optimal control problem with indefinite 
costs, while problem (P2) is an LQ problem with positive costs similar to those encountered 
in the study of target tracking. While problem (P2) is always well-posed and always admits 
an optimal control, problem (PI) will be in general only locally well-posed, and global well- 
posedness will follow from additional assumptions on the parameters of the problem. For more 
details on LQ problems with indefinite costs in infinite dimensions, see |14j . 

Remark 4. The objective functional of problem (PI) could be taken, more generally, as 

rT 

J,{u) = -{Pox{T),x{T))+ / \u{t)\^dt, 

Jo 

with Pq : X ^ X. For example, if the goodwill is more valued in a region Hi C S, Pq could 
be an operator such that Pq = 71/ on H \ Hi, with 7 < 71. Even more generally, one could fix 
a bounded function p : 3 ^> M+ modeling the importance of goodwill at each point of H, and 
define Pq = pi, I being the identity function on X. Note that the above also applies to problem 
(P2). 

4 Solution of the general constrained problem 

The basic idea is to embed the state and control constraints into the structure of the problem. 
Observe that the non-negativity of the intial condition and of the control in ([9]) implies the 
non-negativity of the state variable at any point in time. Using this observation and assigning 
infinite cost to the controls that do not satisfy the constraint < u < Rwe enforce the required 
state and control constraints. Next we show that the problem admits a solution, i.e. that 
the optimal control exists, and we write a weak maximum principle that gives a (abstract) 
characterization of the sought optimal advertising policy. Finally we discuss two special cases 
of the problem for which we can find explicit solutions from the abstract maximum principle. 
Let us define h : X ^>M[J {+00} as follows: 

{/ii(n), u € [0,R] a.e., 
(14) 
+00, otherwise. 



Proposition 5. Assume that 

(i) /iQ : M ^ M is convex and lower semicontinuous; 

(ii) (po is continuous and concave; 

(iii) xq is nonnegative and xq G Hq (H). 
Then there exists at least one optimal pair (x^,,u^,) for problem (P), with x* G C([0,T],X) and 

The proof of this proposition is essentially classical, but we report it for the reader's conve- 
nience and for non-specialists in infinite dimensional control theory. 

Proof. First note that it C Li([0,T],X) n L2([0,T],X) because H is a bounded set in M^ For 
every u G Li([0,T],X) the state system ([9]) admits a unique mild solution x" G C([0, T],X). 
Define the function 

$:L2([0,r],X) ^ M 

u ^ [ h{u{t))dt + <i){x''{T)). 
Jo 

Problem (P) can be equivalently written as 

inf <1> (u) . 

«GL2([0,T],X) 

Let us now show that ^ is convex and lower semicontinous: for A G [0,1] and ui, U2 G 
L2([0,T],X), the convexity of h implies 

/ h{\ui{t) + {I - \)u2{t)) dt < \ I ui{t) dt + {1 - X) U2{t)dt. (15) 

Jo Jo Jo 

One also has 

r-T 



^A«i + (1-A)«2(^) = e-^^xo 



+ / e-^(^-*)5(Ani(t) + (1 - A)n2(t)) dt 
Jo 



Xe-^^xo + (1 - X)e-^^xo 

cT 



+X [ e-^'^^-'^Bui{t)dt + {l-X) [ e~^^'^-'^Bu2{t)dt 
Jo Jo 

X(e-^^x^+ r e-^^^-'^Bu^{t)dt) 

+(1 - A) (e-^^xo + f e-^(^-*)i?n2(t) dt) 



= Ax"nr) + (l-A)x"2(T), 
hence, by the convexity of 0, 

Inequalities ([TC]) and (fTH|) imply 

$(Ani + (1 - X)U2) < X<^{ui) + (1 - A)$(n2), 



i.e. ^ is convex. In order to prove lower semicontinuity, let {un)n>o be a sequence which 
converges strongly to u in L2([0,T] x H). Then Un ^ u in measure, and in particular there 
exists a subsequence n„j. such that Un^ — > u (t,^)-a.e.. By well-known properties of convex 
functions, h is bounded from below by an affine function, i.e. there exists a, 6 G M such that 
h{x) > ax + b, hence 

lim h{unu) — aunu — b = h{u) — au — b (t, ^)-a.e., 
and by Fatou's lemma 

liminf / {h{un^) — aun^. — b)d^dt > / {h{u) — au — b) d^dt, 

rik-^Qo J[o,T]xX J[0,T]xX 

i.e. 

liminf / {h{un^ — h{u)) d^ dt + a liminf / (n — Un^) d^ dt > 0. 

"fe^«2 J[0,T]xX "■fc^°° ^[0,T]xX 

But 



lim inf / {u — Un^ ) d$^ dt 



< lim inf / |n — n„^, | d^ dt 



rife— >oo 

= 0, 



/ \u-Unj'^dCdt 



'[0,T]xX 



1/2 



hence 

liminf/ h{un^{t)) dt > / h{u{t))dt. 

Completely similar arguments show that liminf„j.^oo 0(a;""* (T) > (j){x'^{T)), thus we have 
proved that $ is lower semicontinuous. 

Moreover, one has lim^u^^^ $(n) = +oo. Let now a G M be any (fixed) number. Set E = {x £ 
X : <I>(x) < a}. Then one has 

inf $(x) = inf $(x). (17) 

x£X xGE 

The lower semicontinuity of <1> implies that the level set E is closed, hence weakly closed. 
Moreover, since limui^Qo <l>(n) = +oo, one also has that E is bounded. Therefore E is also 
compact in the weak topology of X. By a well known result, every lower semicontinuous 
function on a compact subset of a topological space attains its infimum. The existence of a 
minimizer now follows immediately from (|17p . 

Let ii# be the minimizer of <1>. Then it is clear that it must be u^ € il. By lemma [2] we also 
have that e is a positivity preserving semigroup. If u* > a.e., then one also has i?n^, > 
a.e. by the assumptions on ^ i— ;■ 6(^). This implies, together with (fTTI) . assumption (iii), and 
the positivity preserving property of e*"^, that x* > a.e., which concludes the proof. D 

Note that this proof of existence of an optimal pair can be adapted to allow for more general 
type of constraints. Consider for instance the problem 



sup</.(x«(r)), 



where u satisfies the budget constraint 

rT 



I l^u\t,Od^<M, 



with M a fixed positive number, and (/> is a concave function. Here we have followed [TC] in 
using the quadratic form for the cost as a function of advertising effort. Then we define the set 
of admissible controls lA C ^2([0, T],X) as 

U=lu: I f u^{t,^)dC<M\ , 

and the functional $ : L2{[0,T],X) ^ M as 

$ : n ^ -0(x"(T)). 
Now we can reformulate the problem as 

inf ^(u). 

In complete analogy to the proof of proposition O one could show that the set U is convex, 
closed, and bounded in L2([0,T],X), that the function $ is convex lower semicontinuous in 
-L2([0,T],X), and that these conditions are enough to ensure that an optimal pair (x*,m*) 
exists, with u^ &U. 

Optimal pairs in many cases can be characterized by the first-order necessary optimality 
conditions, through the introduction of a Lagrange multiplier. In particular, one has to solve 



inf 

ueL2{[0,T],X) 



{x{T)) + X I u^{t)dt 
Jo 



Let ux be the optimal control for this problem. Then n^, is given by a ux such that 

u\{t)dt = M. 



Once the existence of an optimal solution is established, we can characterize optimal pairs 
using the maximum principle in Hilbert spaces. 

Proposition 6. Let (x*,n*) he an optimal pair for problem (P). Then there exists p E C([0, r],X) 
such that x^:, -u*, p satisfy the following two-point boundary value problem: 



x^ 



Ax^. + Bu^ 



(18) 



p' + Ap = 0, 
x*(0) = xo, 
p{T) G -d4>{x.{T)), 
u^t) edh*{Bp{t)), 

where h* is the conjugate of h, and ■' stands for differentiation with respect to time. 

Proof. In order to apply the sufficient and necessary optimality system of Theorem 4.2.1 of [3], 
we need to check some conditions. In particular, A is the generator of a strongly continuous 
semigroup (lemma[2]), and i? is a linear bounded operator, as follows from its definition (jlOp and 
the boundedness of h. The continuity and convexity of (p follow immediately by its definition 
()13p . and similarly for the convexity and lower semicontinuity of /i, cf. ()14p . Therefore the results 
of the theorem hold, from which we get our optimality conditions (jlSp by obvious modifications 
of those in [5] . D 

In the next section we consider two special cases of the general spatial advertising problem 
for which we can provide more specific characterizations of the optimal advertising policies. 
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4.1 Two cases with explicit solutions 

Deriving explicit expressions for the optimal pair from the maximum principle is in general a 
very challenging task. On the other hand, for some specific choices of the objective function, 
these closed-form solutions can be obtained. Consider, for instance, the case where 4){x) = —x, 
and h[u) = |up/2, for u £ [0, R] a.e., and h[u) = +(X) otherwise. Then one has —dcj) = 1, and 



h*{c) 



r 0, c < 0, 

I, (R-R^, C> R- 
Therefore the maximum principle (jlSp yields 

p' + Ap = 0, p{T) = 1, 
hence, by well known properties of the heat equation with Neumann boundary conditions, 

p{t,0 = e-''^^-'\ (t,Oe[0,T]xH. 

From this representation of the dual arc p we immediately obtain that the optimal strategy is 
given by 

u,{t, = h{i)e-P^'^'^'^ A R, a.e.-(t, ^ G [0, T] x H. (19) 

In particular, u^, is always nonnegative and is zero where the advertising effectiveness h is zero. 
Finally, the optimal trajectory x* is given by 



x,(t) = e*%+ / e^*-'^^Bu^{s)ds. 
Jo 



An equivalent expression for the optimal trajectory that is easier to implement numerically can 
be given by projecting on a basis of -/^2(^) of eigenvectors of A. In fact, it is well known (see e.g. 
\ll) that there exists a complete orthonormal (ONC) system {ek)k€N in -^2(S) and a sequence 
of positive numbers {Xk)ken T +oo such that 



Then we have 



Aek = -{Xk + p)ek, VfcGN. 

ken 



a.e. in Hy, where 

x';{t) = e-^^^+P^^x''{0)+ I e-(^^+P)(*-")n*^(s,0(^s, 



and x''(0) = (xq, 6^)^2(5), u''{t) = {b{-)u{t,-),ek) l^^^S)- 

An analogous procedure allows one to obtain the optimal advertising policy and the optimal 
state with linear cost, i.e. with h{u) = n for u G [0, .R] a.e., h{u) = +00 otherwise. In particular 



hence 



^*(*'^)-^ R, b{Op{t,0>i, ^''^ 
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which solves the problem. 

Unfortunately we were unable to obtain closed form expressions for the modified version of 
problem (P2) with linearized cost of control. While the general existence criterion can still be 
applied, the two-point boundary value problem that characterizes the optimal pair cannot be 
solved explicitly. In fact, the maximum principle yields 

p' + Ap = 0, 

x*(0)=xo, (22) 

p{T) = -2{x,{T)-k), 

u^t) edh*{Bp{t)), 

where dh* is given by (|2U|) . 

Remark 7. Note that in the absence of the diffusion term, i.e. for A = —p, the optimal strategies 
([19]) and (pT|) would remain unchanged (because the dual arc p would not change), even though 
the corresponding optimal trajectory do not coincide, as a simple calculation reveals. However, 
this if of course not a general phenomenon, but just a consequence of the equality d(j) = — 1 in 
the above examples. For instance, such simplification does not happen in (|22|) . 

In the next section we employ the dynamic programming principle to give a characterization 
of optimal strategies in feedback form. Even though we cannot obtain, in general, completely 
explicit representation for the optimal strategy, we shall at least indicate some schemes to 
approximate the value function associated to the problem. 

5 Optimal feedback strategies 

Let us define the value function 

Vit, y) = inf (4>{x{T)) + / h{u{s)) ds) , (23) 



subject to 



x' = Ax + Bu, s E [t, T], x{t) = y. (24) 



A mild solution of (I24p will be denoted by x^'". Note that, strictly speaking, the value function 
of problem (P) is —V{t,y). However, for consistency with the previous sections, we prefer to 
study the minimization problem ()23p . which is clearly equivalent to problem (P). 

The existence of an optimal pair (x*,«*) for (i23|) is guaranteed by proposition [5l We shall 
now see that the dynamic programming approach allows one to obtain a characterization of 
(x*,n^,) in terms of the solution of an associated Hamilton- Jacobi equation. 

We first prove two simple but important qualitative properties of the value function. 

Proposition 8. The value function V{t,y) is convex and decreasing in y. 

Proof. Let yi, y2 G ^, and assume that ui, U2 are optimal for V{t, yi) and V{t, ^2), respectively. 
Then we have, for any A G [0, 1], 

\V{t,yi) + {l-\)V{t,y2) = A,/.(x^^'"Hr)) + (l-A)0(x^^'"^(r)) 

+ / \h{ui{s))ds+ I {I - \)h{u2{s)) ds 
Jt Jt 

> (/)(AxJ'i'"i(r) + (l-A)2;J'2'«2(2-)) 
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+ / [A/i(ni(s)) + (1 - X)h{u2{s))] ds, 



where the inequality holds because (p is convex. A simple calculation reveals that 

Ax2'i'"i(r) + (1 - A)a;^2'"2(r) = a;^yi+(i-^)3/2.«(T), 
where u = Xui + (1 — X)u2 € il. By the convexity of the cost function h we then have 

XV{t,yi) + {l-X)V{t,y2) > 0(x^^^+(i-^)'^^'"(r)) + ^ h{n{s)) 

cT 



> inf ((bix^y^+'^^-^'^y^^'^m) + / h(u(s))) 
ueu V Jt ^ 

= V{t,Xyi + {l-X)y2). 



Let us now show that V{t, y) is increasing in y: let u^ be an optimal control, so that 

V{t,y) = <P{xy^^*{T)) + j hiu,is))ds, 
and take y > y. Then we have 

Jit,y,u,) = </>(x^'"*(r))+ / h{u,{s))ds 

T 
( 
t ' Jt 

T , rT 



< ,^(e(^-*)^y+ / e'^^-'-'^^Bu4s)ds^+ f h{u,{s)) ds 



= V{t,y), 

where the inequality holds because the semigroup generated by A is positivity preserving (cf. 
lemma[2|), b{^) > .^-a.e., and (j)o is increasing. The proof is completed observing that V{t,y) > 
J{t,y,u^). D 

A combination of the dynamic programming principle and of the abstract maximum principle 
allows one to write a feedback representation of the optimal strategy in terms of the value 
function V. 

Proposition 9. Let (x*, u*) an optimal pair for i23\) . and V : [0,T]x X ^M. the corresponding 
value function. Then the following inclusion holds: 

u^s) edh*{-B*dV{s,x4s))), 

where dV stands for the suhdifferential of the function y h^ V{s,y). 

Proof. Let v € -^i([0, T], X), and let z be a (mild) solution of z' = Az + Bv. Then we can write 

(x.(r) - z{T),p{T)) - {x,{s) - z{s),p{s)) 
rT rT 

(x'^(r) — z'{r),p{r)) dr + (^^(r) — z{r),p'{r)) dr 

J s 
T rT 

{A{x{r) — z{r)),p{r)) dr + / {B{u^{r) — v{r)),p{r)) dr 
+ / {x*{r) — z{r),p'{r)) dr 



J s 

T 



{B{u^{r) — v{r)),p{r)) dr, 
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where the second equahty foUows by definitions of y and z, and the third equahty follows by 
the identity p' € —A*p. By definition of subdifferential we have 

{B{u^:{r) — v{r)),p{r)) > h{u^{r)) - h{v{r)), 

because u*(t) G dh*{B*p{t)) implies B*p{t) G dh{u^{t)). Recalling that p{T) G -d(t){x^{T)) we 
obtain 

-(x*(s)-z(s),p(s)) >(/.(x*(r))+ /" h{u^{r))dr-ip{z{T))- [ h{v{r))dr. 



The dynamic programming principle now implies 

V{s,x^{s)) = (j){x^{T)) + / h{u:^{r)) dr, 

J s 

which together with the previous inequality yields —p{s) E 9y(s,x*(s)). Recalling that u^{s) G 
dh*{B*p{s)) we finally obtain 

u^{s)(^dh*{-B*dV{s,x.,{s))). D 

One would expect that the value function V solves, in a suitable sense, the Hamilton-Jacobi 
equation associated to problem (|23p . which can be written as 



Vs{s,y) + {Ay,Vy{s,y)) + F{vy{s,y)) = Q, v{T,y) = ^{y), (25) 

for s G [t,T] and y G D{A), where F{q) := -h*{-B*q). In fact (see j4j), assuming that F G 
C^{H), V satisfies (!25]l in the sense that there exists 77(5, y) G dV{s, y) for all (s, y) G [0, T] x X 
such that 

r i;,(s, y) + (^y, 77(5, y)) + F(r/(s, y)) = 0, a.e. t G [0, T], y G D{A), 

\ v{T,y) = (t){y). 

In order to approximate the solution of this terminal value problem, one could apply the 
Crandall-Liggett theorem, which ensures that V is the limit of a sequence of implicit difference 
approximations (time-discretization), under some extra assumptions. For instance, if B = I 
and h{z) = |zp, then V^(f, •) — > V{t, •) uniformly in t as e — > 0, where V^(i, •) solves 

^{Vsis,y) - Ve{s-e,y)) + {Ay,VV,{s,y)) + F{VV,{s,y)) = 0, s e]0,T], y G Z)(A), 
V,{s,y) = ^{y), s = 0, y € D{A). 

A formal justification can be given following [3], sec. 4. 

6 Quadratic cost and reward functions 

In this section we solve the optimal control problem (PI) through the dynamic programming 
approach, that is, first we solve the associated operator Riccati equations, and then we show 
that the optimal control «* can be written as a linear feedback of the optimal trajectory x*. 
Let us recall that problem (PI) amounts to minimizing the objective functional 

Ji(n) = -7|x(r)|2+ f |y(i)|2^i^ 
Jo 

subject to the dynamics x' = Ax + Bu, x{0) = xq. 

In this and the following section we also assume B = I, for simplicity. The analysis of the 
more general case B G >C(X) follows essentially the same steps. 

An abstract solution of problem (PI) is given by the following proposition. 
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Proposition 10. Suppose that the parameters of problem (PI) satisfy the inequality 



2p' 
Then for any xq (^ X , Ji admits a unique minimizer n* given by 

u* = l{I - 7L*TLT)~^L*Te-^^xo, 
with Lt defined in \2b]) . 
Proof. Let us define the following linear operator: 

Lt:L2([0,T],X) ^ X 



u 



AT-s)A 



u{s) ds. 



(26) 



Then we have x{T) = e xq + Ltu, and, denoting by || • || the norm in L2([0, T], X), 

J^{u) = {P^x{T),x{T)) + \\uf 

= (Po(e"^^xo + Ltu), e"^^xo + Ltu) + \\uf 



-A*T , 



-AT 



Poe ^^xo,xo) + 2{Poe ^' xq, Ltu) + {PqLtu, Ltu) + \\u\ 



-AT 



AT„ 



e-^' Poe-^' xo, xo) + 2 (L^Poe"^' xo, u) + {{I + L*tPoLt)u, u) 



(27) 



By ([27]) it immediately follows that if J+L^^Po-^T » 0, then the functional Ji : L2([0, T], X) ^ M 
is convex, lower semi-continuous, and such that liin.iy_]^^ Ji{u) = +oo, hence it attains its 
minimum in L2{[0,T],X). Moreover, again as a consequence of ()27p . if ti* is a minimizer of J, 
then one must have 

(/ + L*tPoLt)u, + L*TPoe-^^xo = 0. (28) 

If^ := {I+L^PqLt) is an homeomorphismof L2([0, T],X) (i.e. if ^~^ is also a linear continuous 
operator on L2([0, T],X)), then we deduce from (|28]l that the unique optimal control u^ is given 

by 

-u* = -^ ^L^Poe'^'^xo. 

From the above discussion, in order to prove the proposition, we need to show that under the 
stated hypotheses ^ is uniformly positive definite, and ^^^ is continuous. In order to prove the 
former, write 



{^v,v) 



\vf + {L^PoLtv,v) 
\v\\ —'j{Ltv,Ltv) 



j{L^Ltv,v) 



v\ 



tI-^t^P, 



where we have used the fact that Pq = —jl- We also have 

rT 



\Ltv\ 



-p(T-s)A{T-s) 



v{s) ds 



< 



< 



,-piT-s) 



\v (s)I ds 



Jo 
K,t) 



-MT-s)^^ 



1/2 



1/2 



\V[S] 



ds 



1/2 
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We have used (in order) a standard estimate, the contractivity of the heat semigroup, the 
Cauchy-Schwarz inequahty, and Fubini's theorem for positive integrands. By 

T 


one has {^v,v) > e||t'|P, with e = 1 — ^ (l — e~^^-^). 

In order to prove the continuity of ^, it is enough to recah the following simple fact: if {^v, v) > 
e||u|P for all v E D{^) and ^ is self-adjoint, then ||^~^|[ < e~^. But ^ = I — ^L^Lt is clearly 
self-adjoint, and the proof is finished. D 

This optimal control suffers of two major drawbacks: it is of the open-loop type, and it 
is difficult to write explicitly (in particular, the computation of ^~^ does not seem to be a 
straightforward task). Appealing to the dynamic programming principle, one can obtain a 
more explicit feedback characterization of the optimal policy: 

Proposition 11. // 

1-^(1- e-2^^) > 0, (29) 

then there exists P a C ([0, T] ; C (X)), P{t) = P{t)* for all t E [0, T] which solves the operator 
Riccati equation 

P' = 2AP - PBB*P, 

P(0) = Po = -iL ^ ' 

Moreover, the optimal control is given by n^,(t) = —B*P (T — t) x^{t), where x^{t) is the unique 
(mild) solution of the closed loop equation 

x' = Ax{t)-BB*P{T-t)x{t), 

x{0) = xo, ^""^ 

and the value function can be written as V{t,y) = Ji{u^) = {P{T — t)y,y). 

Proof. The proof follows the classical scheme based on solving the Riccati equation and applying 
a verification theorem (see e.g. [6j or [9]), but one needs to take into account that Pq is negative 
in our case. However, the coercivity of Jj on L2{[0, T],X) implied by (|29p . "saves" the argument 
(see also theorem 9.4.3 of [S]), and in fact already gives existence and uniqueness of the optimal 
pair (ti*,x^,). Here we limit ourselves to sketch the proof only, pointing out where the essential 
difference is with respect to the usual case. The first step consists in proving that the Riccati 
equation ([30]) admits a local solution in the space C([0, r]; C{X)), r < T (in the usual case one 
has S(X), the space of continuous linear symmetric operators, instead of C{X)), that global 
uniqueness holds, and finally that local solutions can be extended to the whole interval [0, T]. 
The second step is to show that the closed-loop equation (J3T|) admits a unique mild solution 
X E C([0, T],X), which follows as in the classical case. Again following the classical case, in the 
third step one obtains the representation 

Jiiu)= I \uit) + B*P{T-t)x{t)\'^dt + {PiT)xo,xo), 
Jo 

hence Ji{u) > {P{T)x,x), for any u E L2{[0,T],X). Therefore, setting u^ = -B*P{T -t)x^{t), 
with Xt, the solution of the closed-loop equation (f3T]) . the proposition is proved. D 
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6.1 An example with explicit solution 

In this section we assume that B = I, for siniphcity, and we show that the Riccati equation of 
Proposition 4 can actually be solved explicitly in this case. In fact, it is well known (see e.g. 
[1]) that there exists a complete orthonormal (ONC) system {ek)ken in -^2(S) and a sequence 
of positive numbers {Xk)keN T +oo such that 

Aek = -Xk^k, V/c G N. 

Then we can "project" the Riccati equation on this ONC system, obtaining the infinite set of 
Cauchy problems 

Pk{0) = -7, 

where Pfc(-) := P{-)ek- It is immediate that qk{t) = — 2Afc is a particular solution for the A;-th 
problem. Then set 

/ ^ 1 1 

Zk{t) 



Pk{t)-q{t) Pk{t) + 2\k 
One easily obtains that Zk satisfies the linear equation 

dzk 



-2XkZk{t) + l, 
at 



whose general solution is given by 



Zk{t) = ^ + Cke-''^''K 
ZAk 



This in turn implies that the general solution for (j32p is given by 



«(*) = -2^'+pX5-MhiF^- P'> 
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and by the initial condition Ck = oA (2X - 1 • 

An explicit expression for the optimal trajectory can also be obtained, again using a projec- 
tion on the orthonormal system {ek)kefi- In particular, one has 

'^ = -X^x\t)+pk{T-t)x\t) 

for each k, hence 

xj(t) = x'=(0)e-^^*e^oPfc(^-«)<^^, (34) 

and 

oo 

x*{t,i) = Y.xl{t)ek{i). 
fe=o 

We can now write explicitly, in terms of the basis {ek)k&h the optimal distributed strategy. 
Namely, 

<{t)=Pk{T-t)xl{t), 

with pk as in ([32]) , and x^ is given by ((Ml) , hence 



.{t,i) = Y,Pk{T -t)xl{t)ek{i). 



fc=i 
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In general it is not possible to determine explicitly the eigenvalues and the corresponding 
eigenfunctions of A for a generic bounded domain H C M^. However, for particular shapes of H 
the eigensystem is well-known, e.g. for H being a rectangle. If H = [0,L] x [0,H], one has 

with 

em,n(4i, 6) = cos -^— sm — ^ 

and 

2 2 

-^'"''^ = 11^ + :^ J'' • 

Moreover, Ao,o = is an eigenvalue with unit eigenvector eo,o = (LH)'^'"^. Hence Aem,n = 

7 Analysis of the targeting formulation 

In the Hilbert space setting introduced in Sectional assuming B = I for simplicity, let us define 
the distance from the target y : [0, T] ^ X as 

y(t,i) = h{i)-x{t,i), 

where h € -^2(S) is the desired configuration of goodwill to reach at time T. Then y is the 
unique mild solution of the following non-homogeneous linear Cauchy problem in X: 

r y' = Ay{t) - u{t) + /, 

< (35) 

[ 2/(0) = /i-xo, 

with / = -Ah. 

Problem (P2) can be rewritten as 



inf 

«ei2([0,T];£) 



|2 , / I„,/+m2 



7|y(T)|^+ / \u{t)\'dt 





(36) 



subject to (j35p . Appealing again to the dynamic programming principle, we can write the 
Riccati equation associated to problem ([36]) as follows: 



r p'{t) = 2AP{t)-p\t), 

and its adjoint backward equation as 

r r'{t) + {A-P{t))r{t) + P{t)f = 0, 

< (38) 

\ r{T) = 0. 

Then one can uniquely solve the tracking problem in terms of the solution to (I37p and (1381) . 

Proposition 12. If the target function h is such that Ah G L2 (H), then the optimal control 
problem (P2) admits a unique optimal control u^ given by the feedback law 

u4t) = P{T-t)y,{t)+r{t), 
18 



where y* is the unique mild solution of the closed-loop equation 

y'(t) = {A- P{T -t))y{t) -r{t) + f, 



fn\ h (39) 



and P{-), r(-) solve, respectively, equations (31) and ^3^) . Moreover, the value function J* is 
given by 

r = j{u^) ■- 



(P(TM0),y(0))+2(r(0) 


y(o)) 


Jo 


Hr{t)J)- 


-|r(t)|2 


dt. 



Proof. It follows essentially the same lines of the "classical" proof in [9j , with the only difference 
that in this case Pq is positive definite (hence we do not need conditions on the data of the 
problem to ensure well-posedness), and we have to take into account a non-homogeneous part 
in the state equation, and consequently of the adjoint equation (|38p accompanying the Riccati 
equation. The assumption Ah € -^2(^) is equivalent to f £ X, so that a (unique) mild solution 
of (j35p exists. For more details we refer to [6]. D 

Note that, in contrast to (PI), there always exists an optimal solution for problem (P2), for 
any choice of the parameters and any initial condition. Below we obtain an expression for the 
optimal trajectory and the optimal control in terms of a basis of L2(^)) as we have done for 
(PI). In particular, by projecting the Riccati equation ([^7|) on the system {ek)keN, we obtain 
the infinite set of Cauchy problems 

^ = -2XkPk{t)-pl{t), 
. Pfc(O) = 7, 
each of which admits the explicit solution 

Pk{t) = -2Xk + 



(2Afc)-i + C7fce~2A.t' 

with Cfe = — 7(2Afc(2Afc + j))^^- As before, we have set Pk{-) '■= P{-)ek- 

The adjoint backward Cauchy problem (|38p can be solved similarly: projecting on the system 
(efc)fcGN we get 

-Tj- ->^krk{t) - Pk{T - t)rk{t) +Pk{T -t)fk = 0, 
rk(T) = 0, 
where rfc(-) := r(-)efe and fk := /e^. Setting r]k{t) := rk(T - t), one has 

-T- = Afcr?fc(t) -Pk{t)Vk{t) +Pk{t)fk, 
%(0) = 0, 

These Cauchy problems can be solved explicitly, yielding 

Jo 
and finally rk{t) = r]k{T — t). 
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The optimal trajectory can also be written explicitly by computing the solution of the 
closed- loop equation. We again project the equation on the system {ek)keN, obtaining 

^ = {-Xk-Pk{T-t))y'^{t)-n{t) + h, 
at 

y''{0)=hk-Xk{0), 



hence 

/o 
where we have set afc(s) := — A^ — pk{T — s). The optimal trajectory can now be written as 



'=(*) = (/ife-x,(0))e/o-'=W'^-+ / e/>'=(^)'^^(/fc-rfc(s))ds, 



y*(i,e) = E^*w^'^(^)- 



A;=0 

Similarly, the optimal policy is given by 



A;=0 A;=0 

8 Discussion 

Our paper extends traditional marketing models of goodwill dynamics to allow for spatially 
distributed advertising. Using an appropriate Hilbert space reformulation, we map a problem 
of profit maximization for a monopoly firm into an optimal control problem in infinite dimensions 
and discuss existence and characterization of its optimal solutions. In some simple, but still 
realistic situations, the optimal strategy can be obtained in closed form. Our analysis provides 
a tractable model to investigate how advertising funds should be distributed over multiple 
markets. 

Some important issues to be considered as follow-up investigations are, for instance, an 
assessment of the empirical adequacy of our model, as well as the analysis of dynamic interac- 
tion between firms advertising in multiple markets. Such analysis would present an important 
complement to the existing literature on single-market advertising competition. 
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